2026-04-10 17:59:33,210 [ 222692 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:53, check_args_and_update_paths) 2026-04-10 17:59:33,210 [ 222692 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:79, check_args_and_update_paths) 2026-04-10 17:59:33,210 [ 222692 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:90, check_args_and_update_paths) 2026-04-10 17:59:33,210 [ 222692 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:92, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_vgayu6 --privileged --dns-search='.' --memory=30709035008 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e CLICKHOUSE_USE_OLD_ANALYZER=1 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel2_0.jsonl --report-log-exclude-logs-on-passed-tests 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_17]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_18]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_19]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_20]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_21]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_22]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_23]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_24]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_25]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_3]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_4]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_5]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_6]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_7]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_80]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_81]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_9]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_default]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_3]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_4]' 'test_mysql57_database_engine/test.py::test_mysql_types[float_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[float_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6]' 'test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default]' test_mysql57_database_engine/test.py::test_password_leak test_mysql57_database_engine/test.py::test_predefined_connection_configuration test_mysql57_database_engine/test.py::test_restart_server test_profile_settings_and_constraints_order/test.py::test_profile_settings_and_constraints_order test_prometheus_endpoint/test.py::test_prometheus_endpoint test_recompression_ttl/test.py::test_recompression_multiple_ttls test_recompression_ttl/test.py::test_recompression_replicated test_recompression_ttl/test.py::test_recompression_simple test_refreshable_mv/test.py::test_adding_replica test_refreshable_mv/test.py::test_backup_inner_table test_refreshable_mv/test.py::test_backup_outer_table test_refreshable_mv/test.py::test_pause test_refreshable_mv/test.py::test_refresh_vs_shutdown_smoke test_refreshable_mv/test.py::test_refreshable_mv_in_read_only_node test_refreshable_mv/test.py::test_refreshable_mv_in_replicated_db test_refreshable_mv/test.py::test_refreshable_mv_in_system_db test_remote_prewhere/test.py::test_remote test_rename_column/test.py::test_rename_distributed test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select test_rename_column/test.py::test_rename_parallel test_rename_column/test.py::test_rename_parallel_same_node test_rename_column/test.py::test_rename_with_parallel_insert test_rename_column/test.py::test_rename_with_parallel_merges test_rename_column/test.py::test_rename_with_parallel_select test_rename_column/test.py::test_rename_with_parallel_slow_insert test_rename_column/test.py::test_rename_with_parallel_ttl_delete test_rename_column/test.py::test_rename_with_parallel_ttl_move test_replicated_merge_tree_thread_schedule_timeouts/test.py::test_query_exception_on_thread_pool_full test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_auxiliary_zookeeper test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_default_zookeeper test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_not_exists_auxiliary_zookeeper test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_drop_replicated_merge_tree_with_auxiliary_zookeeper test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_path_ambiguity test_replication_without_zookeeper/test.py::test_startup_without_zookeeper test_restart_with_unavailable_azure/test.py::test_cluster_alive_after_restart test_restore_external_engines/test.py::test_restore_table test_restore_external_engines/test.py::test_restore_table_null test_rocksdb_read_only/test.py::test_dirctory_missing_after_stop test_rocksdb_read_only/test.py::test_read_only test_role/test.py::test_admin_option test_role/test.py::test_changing_default_roles_affects_new_sessions_only test_role/test.py::test_combine_privileges test_role/test.py::test_create_role test_role/test.py::test_function_current_roles test_role/test.py::test_grant_role_to_role test_role/test.py::test_introspection test_role/test.py::test_revoke_requires_admin_option 'test_role/test.py::test_role_expiration[False]' 'test_role/test.py::test_role_expiration[True]' test_role/test.py::test_roles_cache test_role/test.py::test_set_role 'test_s3_plain_rewritable/test.py::test[cache_s3_plain_rewritable-data/]' 'test_s3_plain_rewritable/test.py::test[s3_plain_rewritable-data/]' 'test_s3_plain_rewritable/test.py::test[s3_plain_rewritable_with_metadata_cache-data_with_cache/]' 'test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3]' 'test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3_with_resolver]' test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy 'test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case1]' 'test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case2]' 'test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case3]' 'test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case4]' -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket= rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: timeout-2.3.1, repeat-0.9.3, order-1.0.0, reportlog-0.4.0, xdist-3.5.0, random-order-1.1.1 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_refreshable_mv/test.py::test_adding_replica test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_no_proxy test_mysql57_database_engine/test.py::test_mysql_types[common_types_17] test_s3_plain_rewritable/test.py::test[cache_s3_plain_rewritable-data/] test_restore_external_engines/test.py::test_restore_table test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_auxiliary_zookeeper test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case1] test_role/test.py::test_admin_option test_rename_column/test.py::test_rename_distributed test_recompression_ttl/test.py::test_recompression_multiple_ttls [gw1] [ 1%] PASSED test_role/test.py::test_admin_option test_role/test.py::test_changing_default_roles_affects_new_sessions_only [gw1] [ 2%] PASSED test_role/test.py::test_changing_default_roles_affects_new_sessions_only test_role/test.py::test_combine_privileges [gw1] [ 3%] PASSED test_role/test.py::test_combine_privileges test_role/test.py::test_create_role [gw6] [ 4%] PASSED test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case1] test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case2] [gw4] [ 5%] PASSED test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_auxiliary_zookeeper test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_default_zookeeper [gw0] [ 6%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_17] test_mysql57_database_engine/test.py::test_mysql_types[common_types_18] [gw1] [ 7%] PASSED test_role/test.py::test_create_role test_role/test.py::test_function_current_roles [gw3] [ 8%] PASSED test_refreshable_mv/test.py::test_adding_replica [gw5] [ 9%] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy test_refreshable_mv/test.py::test_backup_inner_table [gw1] [ 10%] PASSED test_role/test.py::test_function_current_roles [gw0] [ 11%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_18] test_mysql57_database_engine/test.py::test_mysql_types[common_types_19] test_role/test.py::test_grant_role_to_role [gw4] [ 12%] PASSED test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_default_zookeeper test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_not_exists_auxiliary_zookeeper [gw6] [ 13%] PASSED test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case2] test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case3] [gw4] [ 14%] PASSED test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_not_exists_auxiliary_zookeeper test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_drop_replicated_merge_tree_with_auxiliary_zookeeper [gw1] [ 15%] PASSED test_role/test.py::test_grant_role_to_role [gw0] [ 16%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_19] test_mysql57_database_engine/test.py::test_mysql_types[common_types_1] test_role/test.py::test_introspection [gw6] [ 17%] PASSED test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case3] test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case4] [gw5] [ 18%] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list [gw0] [ 19%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_1] test_mysql57_database_engine/test.py::test_mysql_types[common_types_20] [gw4] [ 20%] PASSED test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_drop_replicated_merge_tree_with_auxiliary_zookeeper test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_path_ambiguity [gw6] [ 21%] PASSED test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case4] [gw0] [ 22%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_20] test_mysql57_database_engine/test.py::test_mysql_types[common_types_21] [gw5] [ 23%] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list_no_proxy [gw0] [ 24%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_21] test_mysql57_database_engine/test.py::test_mysql_types[common_types_22] [gw4] [ 25%] PASSED test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_path_ambiguity [gw1] [ 26%] PASSED test_role/test.py::test_introspection test_role/test.py::test_revoke_requires_admin_option [gw0] [ 27%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_22] test_mysql57_database_engine/test.py::test_mysql_types[common_types_23] [gw8] [ 28%] PASSED test_s3_plain_rewritable/test.py::test[cache_s3_plain_rewritable-data/] test_s3_plain_rewritable/test.py::test[s3_plain_rewritable-data/] [gw3] [ 29%] PASSED test_refreshable_mv/test.py::test_backup_inner_table [gw5] [ 30%] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_no_proxy [gw0] [ 31%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_23] test_mysql57_database_engine/test.py::test_mysql_types[common_types_24] test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3] test_refreshable_mv/test.py::test_backup_outer_table test_profile_settings_and_constraints_order/test.py::test_profile_settings_and_constraints_order [gw7] [ 32%] PASSED test_recompression_ttl/test.py::test_recompression_multiple_ttls test_recompression_ttl/test.py::test_recompression_replicated [gw9] [ 33%] PASSED test_restore_external_engines/test.py::test_restore_table test_restore_external_engines/test.py::test_restore_table_null [gw0] [ 34%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_24] test_mysql57_database_engine/test.py::test_mysql_types[common_types_25] [gw1] [ 35%] PASSED test_role/test.py::test_revoke_requires_admin_option test_role/test.py::test_role_expiration[False] [gw0] [ 36%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_25] test_mysql57_database_engine/test.py::test_mysql_types[common_types_2] [gw5] [ 37%] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy [gw0] [ 38%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_2] test_mysql57_database_engine/test.py::test_mysql_types[common_types_3] [gw4] [ 39%] PASSED test_profile_settings_and_constraints_order/test.py::test_profile_settings_and_constraints_order [gw1] [ 40%] PASSED test_role/test.py::test_role_expiration[False] test_role/test.py::test_role_expiration[True] [gw0] [ 41%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_3] test_mysql57_database_engine/test.py::test_mysql_types[common_types_4] [gw5] [ 42%] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy test_prometheus_endpoint/test.py::test_prometheus_endpoint [gw0] [ 43%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_4] test_mysql57_database_engine/test.py::test_mysql_types[common_types_5] [gw7] [ 44%] FAILED test_recompression_ttl/test.py::test_recompression_replicated test_recompression_ttl/test.py::test_recompression_simple [gw0] [ 45%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_5] test_mysql57_database_engine/test.py::test_mysql_types[common_types_6] [gw6] [ 46%] PASSED test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3] test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3_with_resolver] [gw1] [ 47%] PASSED test_role/test.py::test_role_expiration[True] test_role/test.py::test_roles_cache [gw0] [ 48%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_6] test_mysql57_database_engine/test.py::test_mysql_types[common_types_7] [gw9] [ 49%] PASSED test_restore_external_engines/test.py::test_restore_table_null [gw2] [ 50%] FAILED test_rename_column/test.py::test_rename_distributed test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select test_replicated_merge_tree_thread_schedule_timeouts/test.py::test_query_exception_on_thread_pool_full [gw0] [ 51%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_7] test_mysql57_database_engine/test.py::test_mysql_types[common_types_80] [gw4] [ 52%] PASSED test_prometheus_endpoint/test.py::test_prometheus_endpoint [gw6] [ 53%] PASSED test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3_with_resolver] [gw0] [ 54%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_80] test_mysql57_database_engine/test.py::test_mysql_types[common_types_81] test_rocksdb_read_only/test.py::test_dirctory_missing_after_stop [gw0] [ 55%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_81] test_mysql57_database_engine/test.py::test_mysql_types[common_types_9] [gw8] [ 56%] PASSED test_s3_plain_rewritable/test.py::test[s3_plain_rewritable-data/] test_s3_plain_rewritable/test.py::test[s3_plain_rewritable_with_metadata_cache-data_with_cache/] test_restart_with_unavailable_azure/test.py::test_cluster_alive_after_restart [gw0] [ 57%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_9] test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1] [gw1] [ 58%] PASSED test_role/test.py::test_roles_cache test_role/test.py::test_set_role [gw0] [ 59%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1] test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2] [gw7] [ 60%] PASSED test_recompression_ttl/test.py::test_recompression_simple [gw1] [ 61%] PASSED test_role/test.py::test_set_role test_replication_without_zookeeper/test.py::test_startup_without_zookeeper [gw0] [ 62%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2] test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3] [gw0] [ 63%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3] test_mysql57_database_engine/test.py::test_mysql_types[datetime_default] [gw0] [ 64%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_default] test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1] [gw0] [ 65%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1] test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2] [gw5] [ 66%] PASSED test_replicated_merge_tree_thread_schedule_timeouts/test.py::test_query_exception_on_thread_pool_full [gw0] [ 67%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2] test_mysql57_database_engine/test.py::test_mysql_types[decimal_1] [gw0] [ 68%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_1] test_mysql57_database_engine/test.py::test_mysql_types[decimal_2] [gw9] [ 69%] PASSED test_rocksdb_read_only/test.py::test_dirctory_missing_after_stop test_rocksdb_read_only/test.py::test_read_only [gw0] [ 70%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_2] test_mysql57_database_engine/test.py::test_mysql_types[decimal_3] [gw8] [ 71%] PASSED test_s3_plain_rewritable/test.py::test[s3_plain_rewritable_with_metadata_cache-data_with_cache/] [gw9] [ 72%] PASSED test_rocksdb_read_only/test.py::test_read_only [gw0] [ 73%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_3] test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6] [gw6] [ 74%] PASSED test_replication_without_zookeeper/test.py::test_startup_without_zookeeper [gw0] [ 75%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6] test_mysql57_database_engine/test.py::test_mysql_types[decimal_4] [gw0] [ 76%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_4] test_mysql57_database_engine/test.py::test_mysql_types[float_1] [gw0] [ 77%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[float_1] test_mysql57_database_engine/test.py::test_mysql_types[float_2] [gw0] [ 78%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[float_2] test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6] [gw0] [ 79%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6] test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default] [gw0] [ 80%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default] test_mysql57_database_engine/test.py::test_password_leak [gw0] [ 81%] PASSED test_mysql57_database_engine/test.py::test_password_leak test_mysql57_database_engine/test.py::test_predefined_connection_configuration test_remote_prewhere/test.py::test_remote [gw0] [ 82%] PASSED test_mysql57_database_engine/test.py::test_predefined_connection_configuration test_mysql57_database_engine/test.py::test_restart_server [gw4] [ 83%] PASSED test_restart_with_unavailable_azure/test.py::test_cluster_alive_after_restart [gw8] [ 84%] PASSED test_remote_prewhere/test.py::test_remote [gw0] [ 85%] PASSED test_mysql57_database_engine/test.py::test_restart_server [gw3] [ 86%] FAILED test_refreshable_mv/test.py::test_backup_outer_table test_refreshable_mv/test.py::test_pause [gw3] [ 87%] PASSED test_refreshable_mv/test.py::test_pause test_refreshable_mv/test.py::test_refresh_vs_shutdown_smoke [gw2] [ 88%] FAILED test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select test_rename_column/test.py::test_rename_parallel [gw2] [ 89%] SKIPPED test_rename_column/test.py::test_rename_parallel test_rename_column/test.py::test_rename_parallel_same_node [gw2] [ 90%] SKIPPED test_rename_column/test.py::test_rename_parallel_same_node test_rename_column/test.py::test_rename_with_parallel_insert [gw2] [ 91%] SKIPPED test_rename_column/test.py::test_rename_with_parallel_insert test_rename_column/test.py::test_rename_with_parallel_merges [gw2] [ 92%] SKIPPED test_rename_column/test.py::test_rename_with_parallel_merges test_rename_column/test.py::test_rename_with_parallel_select [gw2] [ 93%] SKIPPED test_rename_column/test.py::test_rename_with_parallel_select test_rename_column/test.py::test_rename_with_parallel_slow_insert [gw2] [ 94%] SKIPPED test_rename_column/test.py::test_rename_with_parallel_slow_insert test_rename_column/test.py::test_rename_with_parallel_ttl_delete [gw2] [ 95%] SKIPPED test_rename_column/test.py::test_rename_with_parallel_ttl_delete test_rename_column/test.py::test_rename_with_parallel_ttl_move [gw2] [ 96%] SKIPPED test_rename_column/test.py::test_rename_with_parallel_ttl_move [gw3] [ 97%] PASSED test_refreshable_mv/test.py::test_refresh_vs_shutdown_smoke test_refreshable_mv/test.py::test_refreshable_mv_in_read_only_node [gw3] [ 98%] PASSED test_refreshable_mv/test.py::test_refreshable_mv_in_read_only_node test_refreshable_mv/test.py::test_refreshable_mv_in_replicated_db [gw3] [ 99%] PASSED test_refreshable_mv/test.py::test_refreshable_mv_in_replicated_db test_refreshable_mv/test.py::test_refreshable_mv_in_system_db [gw3] [100%] PASSED test_refreshable_mv/test.py::test_refreshable_mv_in_system_db Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.10/multiprocessing/pool.py", line 271, in __del__ self._change_notifier.put(None) File "/usr/lib/python3.10/multiprocessing/queues.py", line 377, in put self._writer.send_bytes(obj) File "/usr/lib/python3.10/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/usr/lib/python3.10/multiprocessing/connection.py", line 411, in _send_bytes self._send(header + buf) File "/usr/lib/python3.10/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) OSError: [Errno 9] Bad file descriptor =================================== FAILURES =================================== ________________________ test_recompression_replicated _________________________ [gw7] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = def test_recompression_replicated(started_cluster): for i, node in enumerate([node1, node2]): > node.query( "CREATE TABLE recompression_replicated (d DateTime, key UInt64, data String) \ ENGINE ReplicatedMergeTree('/test/rr', '{}') ORDER BY tuple() \ TTL d + INTERVAL 10 SECOND RECOMPRESS CODEC(ZSTD(13)) SETTINGS merge_with_recompression_ttl_timeout = 0".format( i + 1 ) ) test_recompression_ttl/test.py:165: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3659: in query return self.client.query( helpers/client.py:39: in wrap return func(self, *args, **kwargs) helpers/client.py:79: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 231, stderr: Received exception from server (version 25.3.8): E Code: 999. DB::Exception: Received from 172.16.7.6:9000. Coordination::Exception. Coordination::Exception: Session expired. Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x0000000036f39bb1 E 1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001b1bca51 E 2. ./src/Common/Exception.h:112: Coordination::Exception::Exception(T&&, Coordination::Error) @ 0x000000003073bb0d E 3. ./src/Common/ZooKeeper/KeeperException.h:58: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x00000000307cb1f3 E 4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1651: Coordination::ZooKeeper::multi(std::span const, 18446744073709551615ul>, std::function) @ 0x00000000307d86e7 E 5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:1449: zkutil::ZooKeeper::asyncTryMultiNoThrow(std::span const, 18446744073709551615ul>) @ 0x00000000306e29c4 E 6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:1436: zkutil::ZooKeeper::multiImpl(std::vector, std::allocator>> const&, std::vector, std::allocator>>&, bool) @ 0x00000000306c7db7 E 7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:462: zkutil::ZooKeeper::createAncestors(String const&) @ 0x00000000306c6bb5 E 8. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:936: DB::StorageReplicatedMergeTree::createTableIfNotExistsAttempt(std::shared_ptr const&, std::shared_ptr) const @ 0x000000002d8ff33a E 9. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:924: DB::StorageReplicatedMergeTree::createTableIfNotExists(std::shared_ptr const&, DB::ZooKeeperRetriesInfo const&) const @ 0x000000002d8e7394 E 10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:588: DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(DB::TableZnodeInfo const&, DB::LoadingStrictnessLevel, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>, bool, DB::ZooKeeperRetriesInfo const&) @ 0x000000002d8e1b26 E 11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:41: std::shared_ptr std::allocate_shared[abi:ne190107], DB::TableZnodeInfo&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, bool&, DB::ZooKeeperRetriesInfo&, 0>(std::allocator const&, DB::TableZnodeInfo&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&, bool&, DB::ZooKeeperRetriesInfo&) @ 0x000000002eaf01e1 E 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:851: DB::create(DB::StorageFactory::Arguments const&) @ 0x000000002eae8f02 E 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002d645bf2 E 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1945: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000297732e8 E 15. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1743: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000002976496d E 16. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:2355: DB::InterpreterCreateQuery::execute() @ 0x000000002977f238 E 17. ./build_docker/./src/Interpreters/executeQuery.cpp:1457: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr&) @ 0x000000002a01c743 E 18. ./build_docker/./src/Interpreters/executeQuery.cpp:1624: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002a014665 E 19. ./build_docker/./src/Server/TCPHandler.cpp:664: DB::TCPHandler::runImpl() @ 0x000000002f19ad76 E 20. ./build_docker/./src/Server/TCPHandler.cpp:2629: DB::TCPHandler::run() @ 0x000000002f1daa5a E 21. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000037114faf E 22. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000037115c77 E 23. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x00000000370244cb E 24. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003701e0c8 E 25. asan_thread_start(void*) @ 0x000000000b7a6e77 E 26. ? @ 0x00007efd44bafac3 E 27. ? @ 0x00007efd44c41850 E . (KEEPER_EXCEPTION) E (query: CREATE TABLE recompression_replicated (d DateTime, key UInt64, data String) ENGINE ReplicatedMergeTree('/test/rr', '1') ORDER BY tuple() TTL d + INTERVAL 10 SECOND RECOMPRESS CODEC(ZSTD(13)) SETTINGS merge_with_recompression_ttl_timeout = 0) helpers/client.py:248: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-10 18:01:06.069000 [ 655 ] DEBUG : Executing query CREATE TABLE recompression_replicated (d DateTime, key UInt64, data String) ENGINE ReplicatedMergeTree('/test/rr', '1') ORDER BY tuple() TTL d + INTERVAL 10 SECOND RECOMPRESS CODEC(ZSTD(13)) SETTINGS merge_with_recompression_ttl_timeout = 0 on node1 (cluster.py:3658, query) ___________________________ test_rename_distributed ____________________________ [gw2] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = def test_rename_distributed(started_cluster): table_name = "test_rename_distributed" try: create_distributed_table(node1, table_name) insert(node1, table_name, 1000) rename_column_on_cluster(node1, table_name, "num2", "foo2") rename_column_on_cluster(node1, "%s_replicated" % table_name, "num2", "foo2") insert(node1, table_name, 1000, col_names=["num", "foo2"]) > select(node1, table_name, "foo2", "1998\n", poll=30) test_rename_column/test.py:741: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ node = table_name = 'test_rename_distributed', col_name = 'foo2' expected_result = '1998\n', iterations = 1, ignore_exception = False slow = False, poll = 30 def select( node, table_name, col_name="num", expected_result=None, iterations=1, ignore_exception=False, slow=False, poll=None, ): for i in range(iterations): start_time = time.time() while True: try: if slow: r = node.query( "SELECT count() FROM (SELECT num2, sleepEachRow(0.5) FROM {} WHERE {} % 1000 > 0) SETTINGS function_sleep_max_microseconds_per_block = 0".format( table_name, col_name ) ) else: r = node.query( "SELECT count() FROM {} WHERE {} % 1000 > 0".format( table_name, col_name ) ) if expected_result: if ( r != expected_result and poll and time.time() - start_time < poll ): continue > assert r == expected_result E AssertionError: assert '1950\n' == '1998\n' E - 1998 E + 1950 test_rename_column/test.py:219: AssertionError ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ------------------------------ Captured log setup ------------------------------ 2026-04-10 17:59:40.566000 [ 640 ] DEBUG : Command:[docker ps | wc -l] (cluster.py:121, run_and_check) 2026-04-10 17:59:40.588000 [ 640 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 17:59:40.588000 [ 640 ] DEBUG : No running containers (conftest.py:95, cleanup_environment) 2026-04-10 17:59:40.588000 [ 640 ] DEBUG : Pruning Docker networks (conftest.py:97, cleanup_environment) 2026-04-10 17:59:40.589000 [ 640 ] DEBUG : Command:[docker network prune --force] (cluster.py:121, run_and_check) 2026-04-10 17:59:40.614000 [ 640 ] DEBUG : Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] (cluster.py:121, run_and_check) 2026-04-10 17:59:40.617000 [ 640 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:145, run_and_check) 2026-04-10 17:59:40.618000 [ 640 ] INFO : Running tests in /ClickHouse/tests/integration/test_rename_column/test.py (cluster.py:2748, start) 2026-04-10 17:59:40.619000 [ 640 ] DEBUG : Cluster start called. is_up=False (cluster.py:2755, start) 2026-04-10 17:59:40.637000 [ 640 ] DEBUG : Docker networks for project roottestrenamecolumn-gw2 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 17:59:40.663000 [ 640 ] DEBUG : Docker containers for project roottestrenamecolumn-gw2 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 17:59:40.691000 [ 640 ] DEBUG : Docker volumes for project roottestrenamecolumn-gw2 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 17:59:40.691000 [ 640 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 17:59:40.710000 [ 640 ] DEBUG : Docker networks for project roottestrenamecolumn-gw2 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 17:59:40.732000 [ 640 ] DEBUG : Docker containers for project roottestrenamecolumn-gw2 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 17:59:40.753000 [ 640 ] DEBUG : Docker volumes for project roottestrenamecolumn-gw2 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 17:59:40.753000 [ 640 ] DEBUG : Command:[docker container list --all --filter name='^/roottestrenamecolumn-gw2-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 17:59:40.778000 [ 640 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 17:59:40.778000 [ 640 ] DEBUG : No running containers for project: roottestrenamecolumn-gw2 (cluster.py:889, cleanup) 2026-04-10 17:59:40.779000 [ 640 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 17:59:40.806000 [ 640 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 17:59:40.806000 [ 640 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 17:59:40.828000 [ 640 ] DEBUG : Stderr:Error response from daemon: a prune operation is already running (cluster.py:147, run_and_check) 2026-04-10 17:59:40.828000 [ 640 ] DEBUG : Exitcode:1 (cluster.py:149, run_and_check) 2026-04-10 17:59:40.828000 [ 640 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 17:59:40.829000 [ 640 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 17:59:40.872000 [ 640 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 17:59:40.872000 [ 640 ] DEBUG : Volumes pruned: 1 (cluster.py:925, cleanup) 2026-04-10 17:59:40.872000 [ 640 ] DEBUG : Setup directory for instance: node1 (cluster.py:2768, start) 2026-04-10 17:59:40.873000 [ 640 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 17:59:40.873000 [ 640 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 17:59:40.873000 [ 640 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 17:59:40.874000 [ 640 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 17:59:40.875000 [ 640 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_rename_column/configs/remote_servers.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/instant_moves.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/part_log.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/zookeeper_session_timeout.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/storage_configuration.xml'] to /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node1/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 17:59:40.877000 [ 640 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node1/database (cluster.py:4772, create_dir) 2026-04-10 17:59:40.877000 [ 640 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node1/logs (cluster.py:4783, create_dir) 2026-04-10 17:59:40.877000 [ 640 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 17:59:40.877000 [ 640 ] DEBUG : Setup directory for instance: node2 (cluster.py:2768, start) 2026-04-10 17:59:40.878000 [ 640 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 17:59:40.878000 [ 640 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 17:59:40.879000 [ 640 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 17:59:40.879000 [ 640 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 17:59:40.880000 [ 640 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_rename_column/configs/remote_servers.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/instant_moves.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/part_log.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/zookeeper_session_timeout.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/storage_configuration.xml'] to /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node2/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 17:59:40.880000 [ 640 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node2/database (cluster.py:4772, create_dir) 2026-04-10 17:59:40.881000 [ 640 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node2/logs (cluster.py:4783, create_dir) 2026-04-10 17:59:40.881000 [ 640 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 17:59:40.881000 [ 640 ] DEBUG : Setup directory for instance: node3 (cluster.py:2768, start) 2026-04-10 17:59:40.881000 [ 640 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 17:59:40.882000 [ 640 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 17:59:40.882000 [ 640 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 17:59:40.882000 [ 640 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 17:59:40.883000 [ 640 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_rename_column/configs/remote_servers.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/instant_moves.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/part_log.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/zookeeper_session_timeout.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/storage_configuration.xml'] to /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node3/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 17:59:40.884000 [ 640 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node3/database (cluster.py:4772, create_dir) 2026-04-10 17:59:40.884000 [ 640 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node3/logs (cluster.py:4783, create_dir) 2026-04-10 17:59:40.884000 [ 640 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 17:59:40.884000 [ 640 ] DEBUG : Setup directory for instance: node4 (cluster.py:2768, start) 2026-04-10 17:59:40.885000 [ 640 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 17:59:40.885000 [ 640 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 17:59:40.885000 [ 640 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 17:59:40.886000 [ 640 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 17:59:40.886000 [ 640 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_rename_column/configs/remote_servers.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/instant_moves.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/part_log.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/zookeeper_session_timeout.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/storage_configuration.xml'] to /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node4/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 17:59:40.887000 [ 640 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node4/database (cluster.py:4772, create_dir) 2026-04-10 17:59:40.887000 [ 640 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node4/logs (cluster.py:4783, create_dir) 2026-04-10 17:59:40.887000 [ 640 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 17:59:40.888000 [ 640 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'LSAN_OPTIONS': 'suppressions=/etc/clickhouse-server/lsan_suppressions.txt', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:5ccda723c1fc', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/.env (cluster.py:96, _create_env_file) 2026-04-10 17:59:40.888000 [ 640 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-10 17:59:40.888000 [ 640 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-10 17:59:40.888000 [ 640 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-10 17:59:40.889000 [ 640 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-10 17:59:40.899000 [ 640 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 826 (connectionpool.py:547, _make_request) 2026-04-10 17:59:40.900000 [ 640 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/.env --project-name roottestrenamecolumn-gw2 --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node4/docker-compose.yml pull] (cluster.py:121, run_and_check) 2026-04-10 17:59:51.563000 [ 640 ] DEBUG : Stderr: node1 Skipped - Image is already being pulled by node4 (cluster.py:147, run_and_check) 2026-04-10 17:59:51.563000 [ 640 ] DEBUG : Stderr: zoo3 Skipped - Image is already being pulled by node4 (cluster.py:147, run_and_check) 2026-04-10 17:59:51.563000 [ 640 ] DEBUG : Stderr: zoo1 Skipped - Image is already being pulled by node4 (cluster.py:147, run_and_check) 2026-04-10 17:59:51.563000 [ 640 ] DEBUG : Stderr: zoo2 Skipped - Image is already being pulled by node4 (cluster.py:147, run_and_check) 2026-04-10 17:59:51.563000 [ 640 ] DEBUG : Stderr: node2 Skipped - Image is already being pulled by node4 (cluster.py:147, run_and_check) 2026-04-10 17:59:51.563000 [ 640 ] DEBUG : Stderr: node3 Skipped - Image is already being pulled by node4 (cluster.py:147, run_and_check) 2026-04-10 17:59:51.563000 [ 640 ] DEBUG : Stderr: node4 Pulling (cluster.py:147, run_and_check) 2026-04-10 17:59:51.563000 [ 640 ] DEBUG : Stderr: node4 Pulled (cluster.py:147, run_and_check) 2026-04-10 17:59:51.564000 [ 640 ] DEBUG : Setup ZooKeeper (cluster.py:2809, start) 2026-04-10 17:59:51.564000 [ 640 ] DEBUG : Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper1/log', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper1/config', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper1/coordination', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper2/log', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper2/config', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper2/coordination', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper3/log', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper3/config', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/keeper3/coordination'] (cluster.py:2810, start) 2026-04-10 17:59:51.565000 [ 640 ] DEBUG : Command:[docker compose --project-name roottestrenamecolumn-gw2 --env-file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] (cluster.py:121, run_and_check) 2026-04-10 17:59:54.499000 [ 640 ] DEBUG : Stderr:time="2026-04-10T17:59:51Z" level=trace msg="Docker Desktop integration not enabled" (cluster.py:147, run_and_check) 2026-04-10 17:59:54.499000 [ 640 ] DEBUG : Stderr: Network roottestrenamecolumn-gw2_default Creating (cluster.py:147, run_and_check) 2026-04-10 17:59:54.499000 [ 640 ] DEBUG : Stderr: Network roottestrenamecolumn-gw2_default Created (cluster.py:147, run_and_check) 2026-04-10 17:59:54.499000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo1-1 Creating (cluster.py:147, run_and_check) 2026-04-10 17:59:54.500000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo2-1 Creating (cluster.py:147, run_and_check) 2026-04-10 17:59:54.500000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo3-1 Creating (cluster.py:147, run_and_check) 2026-04-10 17:59:54.500000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo2-1 Created (cluster.py:147, run_and_check) 2026-04-10 17:59:54.500000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo1-1 Created (cluster.py:147, run_and_check) 2026-04-10 17:59:54.500000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo3-1 Created (cluster.py:147, run_and_check) 2026-04-10 17:59:54.500000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo1-1 Starting (cluster.py:147, run_and_check) 2026-04-10 17:59:54.500000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo2-1 Starting (cluster.py:147, run_and_check) 2026-04-10 17:59:54.500000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo3-1 Starting (cluster.py:147, run_and_check) 2026-04-10 17:59:54.500000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo1-1 Started (cluster.py:147, run_and_check) 2026-04-10 17:59:54.500000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo3-1 Started (cluster.py:147, run_and_check) 2026-04-10 17:59:54.500000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo2-1 Started (cluster.py:147, run_and_check) 2026-04-10 17:59:54.501000 [ 640 ] DEBUG : Stderr:time="2026-04-10T17:59:54Z" level=debug msg="otel error" error="" (cluster.py:147, run_and_check) 2026-04-10 17:59:54.501000 [ 640 ] DEBUG : Stderr:time="2026-04-10T17:59:54Z" level=debug msg="otel error" error="" (cluster.py:147, run_and_check) 2026-04-10 17:59:54.501000 [ 640 ] DEBUG : Wait ZooKeeper to start (cluster.py:2446, wait_zookeeper_to_start) 2026-04-10 17:59:54.501000 [ 640 ] DEBUG : get_instance_ip instance_name=zoo1 (cluster.py:2015, get_instance_ip) 2026-04-10 17:59:54.507000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-zoo1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 17:59:54.509000 [ 640 ] DEBUG : get_kazoo_client: zoo1, ip:172.16.9.4, port:2181, use_ssl:False (cluster.py:3322, get_kazoo_client) 2026-04-10 17:59:54.511000 [ 640 ] INFO : Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 17:59:54.511000 [ 640 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 17:59:54.608000 [ 640 ] INFO : Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 17:59:54.608000 [ 640 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 17:59:54.764000 [ 640 ] INFO : Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 17:59:54.765000 [ 640 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 17:59:55.070000 [ 640 ] INFO : Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 17:59:55.070000 [ 640 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 17:59:55.814000 [ 640 ] INFO : Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 17:59:55.815000 [ 640 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 17:59:56.752000 [ 640 ] INFO : Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 17:59:56.752000 [ 640 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 17:59:58.342000 [ 640 ] INFO : Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 17:59:58.342000 [ 640 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-04-10 17:59:58.360000 [ 640 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-04-10 17:59:58.361000 [ 640 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-04-10 17:59:58.364000 [ 640 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-04-10 17:59:58.365000 [ 640 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-04-10 17:59:58.374000 [ 640 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-04-10 17:59:58.374000 [ 640 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-04-10 17:59:58.374000 [ 640 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-04-10 17:59:58.465000 [ 640 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-04-10 17:59:58.468000 [ 640 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-04-10 17:59:58.468000 [ 640 ] DEBUG : get_instance_ip instance_name=zoo2 (cluster.py:2015, get_instance_ip) 2026-04-10 17:59:58.484000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-zoo2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 17:59:58.488000 [ 640 ] DEBUG : get_kazoo_client: zoo2, ip:172.16.9.2, port:2181, use_ssl:False (cluster.py:3322, get_kazoo_client) 2026-04-10 17:59:58.498000 [ 640 ] INFO : Connecting to 172.16.9.2(172.16.9.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 17:59:58.500000 [ 640 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-04-10 17:59:58.517000 [ 640 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-04-10 17:59:58.518000 [ 640 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-04-10 17:59:58.518000 [ 640 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-04-10 17:59:58.519000 [ 640 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-04-10 17:59:58.524000 [ 640 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-04-10 17:59:58.525000 [ 640 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-04-10 17:59:58.525000 [ 640 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-04-10 17:59:58.594000 [ 640 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-04-10 17:59:58.594000 [ 640 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-04-10 17:59:58.594000 [ 640 ] DEBUG : get_instance_ip instance_name=zoo3 (cluster.py:2015, get_instance_ip) 2026-04-10 17:59:58.609000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-zoo3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 17:59:58.609000 [ 640 ] DEBUG : get_kazoo_client: zoo3, ip:172.16.9.3, port:2181, use_ssl:False (cluster.py:3322, get_kazoo_client) 2026-04-10 17:59:58.611000 [ 640 ] INFO : Connecting to 172.16.9.3(172.16.9.3):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 17:59:58.611000 [ 640 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-04-10 17:59:58.617000 [ 640 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-04-10 17:59:58.619000 [ 640 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-04-10 17:59:58.621000 [ 640 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-04-10 17:59:58.622000 [ 640 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-04-10 17:59:58.636000 [ 640 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-04-10 17:59:58.637000 [ 640 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-04-10 17:59:58.637000 [ 640 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-04-10 17:59:58.737000 [ 640 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-04-10 17:59:58.738000 [ 640 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-04-10 17:59:58.738000 [ 640 ] DEBUG : All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') (cluster.py:2462, wait_zookeeper_nodes_to_start) 2026-04-10 17:59:58.738000 [ 640 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/.env --project-name roottestrenamecolumn-gw2 --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node4/docker-compose.yml up -d --no-recreate') (cluster.py:3149, start) 2026-04-10 17:59:58.739000 [ 640 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/.env --project-name roottestrenamecolumn-gw2 --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw2/node4/docker-compose.yml up -d --no-recreate] (cluster.py:121, run_and_check) 2026-04-10 18:00:00.403000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo3-1 Running (cluster.py:147, run_and_check) 2026-04-10 18:00:00.404000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo1-1 Running (cluster.py:147, run_and_check) 2026-04-10 18:00:00.404000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-zoo2-1 Running (cluster.py:147, run_and_check) 2026-04-10 18:00:00.404000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node4-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:00:00.404000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node1-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:00:00.404000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node2-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:00:00.404000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node3-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:00:00.404000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node2-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:00:00.404000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node3-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:00:00.404000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node4-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:00:00.404000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node1-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:00:00.404000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node1-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:00:00.405000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node2-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:00:00.405000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node3-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:00:00.405000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node4-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:00:00.405000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node2-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:00:00.405000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node1-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:00:00.405000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node3-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:00:00.405000 [ 640 ] DEBUG : Stderr: Container roottestrenamecolumn-gw2-node4-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:00:00.405000 [ 640 ] DEBUG : ClickHouse instance created (cluster.py:3157, start) 2026-04-10 18:00:00.405000 [ 640 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2015, get_instance_ip) 2026-04-10 18:00:00.415000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:00.416000 [ 640 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:00:00.418000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:00.419000 [ 640 ] DEBUG : Waiting for ClickHouse start in node1, ip: 172.16.9.7... (cluster.py:3165, start) 2026-04-10 18:00:00.430000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:00.434000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:00.540000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:00.654000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:00.760000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:00.864000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:00.967000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:01.076000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:01.182000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:01.289000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:01.394000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:01.497000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:01.601000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:01.705000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:01.809000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:01.915000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.023000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.147000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.251000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.356000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.460000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.564000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.668000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.771000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.874000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/04d5998e02dd5758b487227139bbdb9b229d0d14dcdbee8c955d34f1b172c14c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.875000 [ 640 ] DEBUG : ClickHouse node1 started (cluster.py:3169, start) 2026-04-10 18:00:02.875000 [ 640 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2015, get_instance_ip) 2026-04-10 18:00:02.877000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.878000 [ 640 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:00:02.880000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.881000 [ 640 ] DEBUG : Waiting for ClickHouse start in node2, ip: 172.16.9.8... (cluster.py:3165, start) 2026-04-10 18:00:02.883000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.886000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/966c715fca5f5a26162004f0012f00f2e2c584af7b797b75ea76a1b97b42a6a0/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.887000 [ 640 ] DEBUG : ClickHouse node2 started (cluster.py:3169, start) 2026-04-10 18:00:02.887000 [ 640 ] DEBUG : get_instance_ip instance_name=node3 (cluster.py:2015, get_instance_ip) 2026-04-10 18:00:02.889000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.889000 [ 640 ] DEBUG : get_instance_ip instance_name=node3 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:00:02.891000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.892000 [ 640 ] DEBUG : Waiting for ClickHouse start in node3, ip: 172.16.9.6... (cluster.py:3165, start) 2026-04-10 18:00:02.894000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.897000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/02be58d7ad5075f6a92e6c4ab365125fa31cd2a8c3434b4e46c7a28e111b17e8/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.898000 [ 640 ] DEBUG : ClickHouse node3 started (cluster.py:3169, start) 2026-04-10 18:00:02.898000 [ 640 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2015, get_instance_ip) 2026-04-10 18:00:02.901000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.901000 [ 640 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:00:02.903000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.904000 [ 640 ] DEBUG : Waiting for ClickHouse start in node4, ip: 172.16.9.5... (cluster.py:3165, start) 2026-04-10 18:00:02.906000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw2-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.911000 [ 640 ] DEBUG : http://localhost:None "GET /v1.46/containers/8ee5b80d3e3af882e839cf007b456928e33b8ad999c335bdf099f71262fe1914/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:00:02.912000 [ 640 ] DEBUG : ClickHouse node4 started (cluster.py:3169, start) ------------------------------ Captured log call ------------------------------- 2026-04-10 18:00:02.915000 [ 640 ] DEBUG : Executing query CREATE TABLE test_rename_distributed_replicated ON CLUSTER test_cluster ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/{shard}/test_rename_distributed_replicated', '{replica}') ORDER BY num PARTITION BY num % 100; on node1 (cluster.py:3658, query) 2026-04-10 18:00:03.541000 [ 640 ] DEBUG : Executing query CREATE TABLE test_rename_distributed ON CLUSTER test_cluster AS test_rename_distributed_replicated ENGINE = Distributed(test_cluster, default, test_rename_distributed_replicated, rand()) on node1 (cluster.py:3658, query) 2026-04-10 18:00:03.859000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(1000) on node1 (cluster.py:3658, query) 2026-04-10 18:00:18.985000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 (cluster.py:3658, query) 2026-04-10 18:00:19.589000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_replicated ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 (cluster.py:3658, query) 2026-04-10 18:00:47.566000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(1000) on node1 (cluster.py:3658, query) 2026-04-10 18:01:25.293000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 (cluster.py:3658, query) 2026-04-10 18:01:34.858000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 (cluster.py:3658, query) 2026-04-10 18:01:45.627000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 (cluster.py:3658, query) 2026-04-10 18:01:51.623000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 (cluster.py:3658, query) 2026-04-10 18:01:55.690000 [ 640 ] DEBUG : Executing query DROP TABLE IF EXISTS test_rename_distributed ON CLUSTER test_cluster SYNC on node1 (cluster.py:3658, query) 2026-04-10 18:01:56.446000 [ 640 ] DEBUG : Executing query DROP TABLE IF EXISTS test_rename_distributed_replicated ON CLUSTER test_cluster SYNC on node1 (cluster.py:3658, query) ___________________________ test_backup_outer_table ____________________________ [gw3] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = cleanup = None def test_backup_outer_table(started_cluster, cleanup): > do_test_backup(True) test_refreshable_mv/test.py:444: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test_refreshable_mv/test.py:427: in do_test_backup node1.query(f"RESTORE ALL ON CLUSTER 'default' FROM {backup_destination};") helpers/cluster.py:3659: in query return self.client.query( helpers/client.py:39: in wrap return func(self, *args, **kwargs) helpers/client.py:79: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 96, stderr: Received exception from server (version 25.3.8): E Code: 608. DB::Exception: Received from 172.16.5.10:9000. DB::Exception: Got error from host node1:9000. DB::Exception: Couldn't restore table re.tgt on other node or sync it (elapsed 300.049420s): While creating table re.tgt. Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x0000000036f39bb1 E 1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001b1bca51 E 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000b7f426b E 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000b982fe4 E 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:2038: DB::DatabaseReplicated::createTableRestoredFromBackup(std::shared_ptr const&, std::shared_ptr, std::shared_ptr, unsigned long) @ 0x0000000027665943 E 5. ./build_docker/./src/Backups/RestorerFromBackup.cpp:1009: DB::RestorerFromBackup::createTable(DB::QualifiedTableName const&) @ 0x0000000026aa02d5 E 6. ./build_docker/./src/Backups/RestorerFromBackup.cpp:951: void std::__function::__policy_invoker::__call_impl[abi:ne190107]>(std::__function::__policy_storage const*) @ 0x0000000026aa8860 E 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x0000000026aa6251 E 8. ./contrib/llvm-project/libcxx/include/future:1589: std::packaged_task::operator()() @ 0x0000000025e4a6cc E 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b48c632 E 10. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::ThreadFromThreadPool::*)(), ThreadPoolImpl>::ThreadFromThreadPool*>(void (ThreadPoolImpl>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl>::ThreadFromThreadPool*&&)::'lambda'()::operator()() @ 0x000000001b49a583 E 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b486e11 E 12. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001b495270 E 13. asan_thread_start(void*) @ 0x000000000b7a6e77 E 14. ? @ 0x00007fa12597bac3 E 15. ? @ 0x00007fa125a0d850 E E Job's origin stack trace: E 0. ./build_docker/./src/Common/StackTrace.cpp:386: StackTrace::StackTrace() @ 0x000000001b3230c7 E 1. ./build_docker/./src/Common/ThreadPool.cpp:130: void boost::heap::priority_queue, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::emplace, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const, bool&, (anonymous namespace)::ScopedDecrement>(std::function&&, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const&&, bool&, (anonymous namespace)::ScopedDecrement&&) @ 0x000000001b4853c4 E 2. ./build_docker/./src/Common/ThreadPool.cpp:401: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001b4900ca E 3. ./build_docker/./src/Common/ThreadPool.cpp:494: ThreadPoolImpl>::scheduleOrThrowOnError(std::function, Priority) @ 0x000000001b48f4f7 E 4. ./src/Common/threadPoolCallbackRunner.h:52: std::future std::__function::__policy_invoker (DB::RestorerFromBackup::schedule(std::function&&, char const*)::$_0&&, Priority)>::__call_impl[abi:ne190107] (DB::RestorerFromBackup::schedule(std::function&&, char const*)::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe&&, char const*)::$_0>(ThreadPoolImpl>&, String const&)::'lambda'(DB::RestorerFromBackup::schedule(std::function&&, char const*)::$_0&&, Priority), std::future (DB::RestorerFromBackup::schedule(std::function&&, char const*)::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::RestorerFromBackup::schedule(std::function&&, char const*)::$_0&&, Priority&&) @ 0x0000000026aa5101 E 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x0000000026a91207 E 6. ./build_docker/./src/Backups/RestorerFromBackup.cpp:943: DB::RestorerFromBackup::createAndCheckTable(DB::QualifiedTableName const&) @ 0x0000000026a9ed0f E 7. ./build_docker/./src/Backups/RestorerFromBackup.cpp:937: DB::RestorerFromBackup::createAndCheckTablesWithSameDependencyLevel(std::vector> const&) @ 0x0000000026a9e8c1 E 8. ./build_docker/./src/Backups/RestorerFromBackup.cpp:931: DB::RestorerFromBackup::createAndCheckTables() @ 0x0000000026a8ffbb E 9. ./build_docker/./src/Backups/RestorerFromBackup.cpp:172: DB::RestorerFromBackup::run(DB::RestorerFromBackup::Mode) @ 0x0000000026a87246 E 10. ./build_docker/./src/Backups/BackupsWorker.cpp:948: DB::BackupsWorker::doRestore(std::shared_ptr const&, String const&, DB::BackupInfo const&, DB::RestoreSettings, std::shared_ptr, std::shared_ptr, std::shared_ptr const&, bool, std::shared_ptr const&) @ 0x0000000026952dcf E 11. ./build_docker/./src/Backups/BackupsWorker.cpp:793: DB::BackupsWorker::RestoreStarter::doRestore() @ 0x000000002696abc0 E 12. ./build_docker/./src/Backups/BackupsWorker.cpp:840: void std::__function::__policy_invoker::__call_impl[abi:ne190107] const&, std::shared_ptr)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000002695bbd3 E 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x0000000025e4a0da E 14. ./contrib/llvm-project/libcxx/include/future:1589: std::packaged_task::operator()() @ 0x0000000025e4a6cc E 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b48c632 E 16. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::ThreadFromThreadPool::*)(), ThreadPoolImpl>::ThreadFromThreadPool*>(void (ThreadPoolImpl>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl>::ThreadFromThreadPool*&&)::'lambda'()::operator()() @ 0x000000001b49a583 E 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b486e11 E 18. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001b495270 E 19. asan_thread_start(void*) @ 0x000000000b7a6e77 E 20. ? @ 0x00007fa12597bac3 E 21. ? @ 0x00007fa125a0d850 E E Job's origin stack trace: E 0. ./build_docker/./src/Common/StackTrace.cpp:386: StackTrace::StackTrace() @ 0x000000001b3230c7 E 1. ./build_docker/./src/Common/ThreadPool.cpp:130: void boost::heap::priority_queue, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::emplace, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const, bool&, (anonymous namespace)::ScopedDecrement>(std::function&&, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const&&, bool&, (anonymous namespace)::ScopedDecrement&&) @ 0x000000001b4853c4 E 2. ./build_docker/./src/Common/ThreadPool.cpp:401: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001b4900ca E 3. ./build_docker/./src/Common/ThreadPool.cpp:494: ThreadPoolImpl>::scheduleOrThrowOnError(std::function, Priority) @ 0x000000001b48f4f7 E 4. ./src/Common/threadPoolCallbackRunner.h:52: std::function (std::function&&, Priority)> DB::threadPoolCallbackRunnerUnsafe>(ThreadPoolImpl>&, String const&)::'lambda'(std::function&&, Priority)::operator()(std::function&&, Priority) @ 0x0000000025e48abc E 5. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149: std::future std::__function::__policy_invoker (std::function&&, Priority)>::__call_impl[abi:ne190107] (std::function&&, Priority)> DB::threadPoolCallbackRunnerUnsafe>(ThreadPoolImpl>&, String const&)::'lambda'(std::function&&, Priority), std::future (std::function&&, Priority)>>(std::__function::__policy_storage const*, std::function&&, Priority&&) @ 0x0000000025e48614 E 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x0000000026947b31 E 7. ./build_docker/./src/Backups/BackupsWorker.cpp:335: DB::BackupsWorker::start(std::shared_ptr const&, std::shared_ptr) @ 0x0000000026946ef2 E 8. ./build_docker/./src/Interpreters/InterpreterBackupQuery.cpp:44: DB::InterpreterBackupQuery::execute() @ 0x000000002a11d93e E 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1457: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr&) @ 0x000000002a01c743 E 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1761: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x000000002a0240df E 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:510: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&, bool) @ 0x0000000028ba87ef E 12. ./build_docker/./src/Interpreters/DDLWorker.cpp:675: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&, bool) @ 0x0000000028ba4479 E 13. ./build_docker/./src/Interpreters/DDLWorker.cpp:453: DB::DDLWorker::scheduleTasks(bool) @ 0x0000000028b9fa99 E 14. ./build_docker/./src/Interpreters/DDLWorker.cpp:1203: DB::DDLWorker::runMainThread() @ 0x0000000028b93c6c E 15. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()::operator()() @ 0x0000000028bc7403 E 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b486e11 E 17. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001b495270 E 18. asan_thread_start(void*) @ 0x000000000b7a6e77 E 19. ? @ 0x00007fa12597bac3 E 20. ? @ 0x00007fa125a0d850 E . Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x0000000036f39bb1 E 1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001b1bca51 E 2. ./src/Common/Exception.h:111: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001b3c2285 E 3. ./build_docker/./src/Backups/BackupCoordinationStageSync.cpp:1450: DB::BackupCoordinationStageSync::parseErrorNode(String const&) const @ 0x0000000026a35bfc E 4. ./build_docker/./src/Backups/BackupCoordinationStageSync.cpp:584: DB::BackupCoordinationStageSync::readCurrentState(std::shared_ptr) @ 0x0000000026a3b744 E 5. ./build_docker/./src/Backups/BackupCoordinationStageSync.cpp:485: DB::BackupCoordinationStageSync::watchingThread() @ 0x0000000026a38419 E 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x0000000025e4a0da E 7. ./contrib/llvm-project/libcxx/include/future:1589: std::packaged_task::operator()() @ 0x0000000025e4a6cc E 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b48c632 E 9. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::ThreadFromThreadPool::*)(), ThreadPoolImpl>::ThreadFromThreadPool*>(void (ThreadPoolImpl>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl>::ThreadFromThreadPool*&&)::'lambda'()::operator()() @ 0x000000001b49a583 E 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b486e11 E 11. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001b495270 E 12. asan_thread_start(void*) @ 0x000000000b7a6e77 E 13. ? @ 0x00007fa12597bac3 E 14. ? @ 0x00007fa125a0d850 E E Job's origin stack trace: E 0. ./build_docker/./src/Common/StackTrace.cpp:386: StackTrace::StackTrace() @ 0x000000001b3230c7 E 1. ./build_docker/./src/Common/ThreadPool.cpp:130: void boost::heap::priority_queue, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::emplace, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const, bool&, (anonymous namespace)::ScopedDecrement>(std::function&&, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const&&, bool&, (anonymous namespace)::ScopedDecrement&&) @ 0x000000001b4853c4 E 2. ./build_docker/./src/Common/ThreadPool.cpp:401: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001b4900ca E 3. ./build_docker/./src/Common/ThreadPool.cpp:494: ThreadPoolImpl>::scheduleOrThrowOnError(std::function, Priority) @ 0x000000001b48f4f7 E 4. ./src/Common/threadPoolCallbackRunner.h:52: std::function (std::function&&, Priority)> DB::threadPoolCallbackRunnerUnsafe>(ThreadPoolImpl>&, String const&)::'lambda'(std::function&&, Priority)::operator()(std::function&&, Priority) @ 0x0000000025e48abc E 5. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149: std::future std::__function::__policy_invoker (std::function&&, Priority)>::__call_impl[abi:ne190107] (std::function&&, Priority)> DB::threadPoolCallbackRunnerUnsafe>(ThreadPoolImpl>&, String const&)::'lambda'(std::function&&, Priority), std::future (std::function&&, Priority)>>(std::__function::__policy_storage const*, std::function&&, Priority&&) @ 0x0000000025e48614 E 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x0000000026a2f338 E 7. ./build_docker/./src/Backups/BackupCoordinationStageSync.cpp:137: DB::BackupCoordinationStageSync::BackupCoordinationStageSync(bool, String const&, String const&, std::vector> const&, bool, DB::BackupConcurrencyCounters&, DB::WithRetries const&, std::function (std::function&&, Priority)>, std::shared_ptr, std::shared_ptr) @ 0x0000000026a2b366 E 8. ./build_docker/./src/Backups/RestoreCoordinationOnCluster.cpp:39: DB::RestoreCoordinationOnCluster::RestoreCoordinationOnCluster(StrongTypedef, DB::UUIDTag> const&, String const&, std::function ()>, DB::BackupKeeperSettings const&, String const&, std::vector> const&, bool, DB::BackupConcurrencyCounters&, std::function (std::function&&, Priority)>, std::shared_ptr) @ 0x0000000026a699cb E 9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:41: DB::BackupsWorker::makeRestoreCoordination(bool, DB::RestoreSettings const&, std::shared_ptr const&) const @ 0x00000000269554a0 E 10. ./build_docker/./src/Backups/BackupsWorker.cpp:791: DB::BackupsWorker::RestoreStarter::doRestore() @ 0x000000002696a920 E 11. ./build_docker/./src/Backups/BackupsWorker.cpp:840: void std::__function::__policy_invoker::__call_impl[abi:ne190107] const&, std::shared_ptr)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000002695bbd3 E 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x0000000025e4a0da E 13. ./contrib/llvm-project/libcxx/include/future:1589: std::packaged_task::operator()() @ 0x0000000025e4a6cc E 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b48c632 E 15. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::ThreadFromThreadPool::*)(), ThreadPoolImpl>::ThreadFromThreadPool*>(void (ThreadPoolImpl>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl>::ThreadFromThreadPool*&&)::'lambda'()::operator()() @ 0x000000001b49a583 E 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b486e11 E 17. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001b495270 E 18. asan_thread_start(void*) @ 0x000000000b7a6e77 E 19. ? @ 0x00007fa12597bac3 E 20. ? @ 0x00007fa125a0d850 E E Job's origin stack trace: E 0. ./build_docker/./src/Common/StackTrace.cpp:386: StackTrace::StackTrace() @ 0x000000001b3230c7 E 1. ./build_docker/./src/Common/ThreadPool.cpp:130: void boost::heap::priority_queue, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::emplace, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const, bool&, (anonymous namespace)::ScopedDecrement>(std::function&&, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const&&, bool&, (anonymous namespace)::ScopedDecrement&&) @ 0x000000001b4853c4 E 2. ./build_docker/./src/Common/ThreadPool.cpp:401: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001b4900ca E 3. ./build_docker/./src/Common/ThreadPool.cpp:494: ThreadPoolImpl>::scheduleOrThrowOnError(std::function, Priority) @ 0x000000001b48f4f7 E 4. ./src/Common/threadPoolCallbackRunner.h:52: std::function (std::function&&, Priority)> DB::threadPoolCallbackRunnerUnsafe>(ThreadPoolImpl>&, String const&)::'lambda'(std::function&&, Priority)::operator()(std::function&&, Priority) @ 0x0000000025e48abc E 5. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:149: std::future std::__function::__policy_invoker (std::function&&, Priority)>::__call_impl[abi:ne190107] (std::function&&, Priority)> DB::threadPoolCallbackRunnerUnsafe>(ThreadPoolImpl>&, String const&)::'lambda'(std::function&&, Priority), std::future (std::function&&, Priority)>>(std::__function::__policy_storage const*, std::function&&, Priority&&) @ 0x0000000025e48614 E 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x0000000026947b31 E 7. ./build_docker/./src/Backups/BackupsWorker.cpp:335: DB::BackupsWorker::start(std::shared_ptr const&, std::shared_ptr) @ 0x0000000026946ef2 E 8. ./build_docker/./src/Interpreters/InterpreterBackupQuery.cpp:44: DB::InterpreterBackupQuery::execute() @ 0x000000002a11d93e E 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1457: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr&) @ 0x000000002a01c743 E 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1624: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002a014665 E 11. ./build_docker/./src/Server/TCPHandler.cpp:664: DB::TCPHandler::runImpl() @ 0x000000002f19ad76 E 12. ./build_docker/./src/Server/TCPHandler.cpp:2629: DB::TCPHandler::run() @ 0x000000002f1daa5a E 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000037114faf E 14. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000037115c77 E 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x00000000370244cb E 16. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003701e0c8 E 17. asan_thread_start(void*) @ 0x000000000b7a6e77 E 18. ? @ 0x00007fa12597bac3 E 19. ? @ 0x00007fa125a0d850 E . (CANNOT_RESTORE_TABLE) E (query: RESTORE ALL ON CLUSTER 'default' FROM S3('http://minio1:9001/root/data/backups/backup2', 'minio', 'ClickHouse_Minio_P@ssw0rd');) helpers/client.py:248: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-10 18:01:04.445000 [ 643 ] DEBUG : Executing query create database re engine = Replicated('/test/re', 'shard1', '{replica}'); on node1 (cluster.py:3658, query) 2026-04-10 18:01:05.294000 [ 643 ] DEBUG : Executing query create database re engine = Replicated('/test/re', 'shard1', '{replica}'); on node2 (cluster.py:3658, query) 2026-04-10 18:01:05.925000 [ 643 ] DEBUG : Executing query create table re.tgt (x Int64) engine ReplicatedMergeTree order by x on node1 (cluster.py:3658, query) 2026-04-10 18:01:10.722000 [ 643 ] DEBUG : Executing query create table re.src (x Int64) engine ReplicatedMergeTree order by x;insert into re.src values (1); on node1 (cluster.py:3658, query) 2026-04-10 18:01:11.862000 [ 643 ] DEBUG : Executing query system sync database replica re;system sync replica re.src;create materialized view re.rmv refresh every 1 second TO re.tgt as select x from re.src;system wait view re.rmv on node2 (cluster.py:3658, query) 2026-04-10 18:01:13.853000 [ 643 ] DEBUG : Executing query select * from re.tgt on node2 (cluster.py:3658, query) 2026-04-10 18:01:14.455000 [ 643 ] DEBUG : Executing query BACKUP ALL EXCEPT DATABASE system ON CLUSTER 'default' TO S3('http://minio1:9001/root/data/backups/backup2', 'minio', 'ClickHouse_Minio_P@ssw0rd'); on node1 (cluster.py:3658, query) 2026-04-10 18:01:34.315000 [ 643 ] DEBUG : Executing query drop database re sync;drop database if exists re_broken_replicated_tables sync; on node1 (cluster.py:3658, query) 2026-04-10 18:01:35.907000 [ 643 ] DEBUG : Executing query drop database re sync;drop database if exists re_broken_replicated_tables sync; on node2 (cluster.py:3658, query) 2026-04-10 18:01:46.135000 [ 643 ] DEBUG : Executing query SELECT count() FROM system.tables where database='re' AND name in ('src', 'rmv') on node1 (cluster.py:3658, query) 2026-04-10 18:01:47.040000 [ 643 ] DEBUG : Executing query RESTORE ALL ON CLUSTER 'default' FROM S3('http://minio1:9001/root/data/backups/backup2', 'minio', 'ClickHouse_Minio_P@ssw0rd'); on node1 (cluster.py:3658, query) ---------------------------- Captured log teardown ----------------------------- 2026-04-10 18:06:58.209000 [ 643 ] DEBUG : Executing query drop database if exists re sync;drop table if exists system.a; on node1 (cluster.py:3658, query) 2026-04-10 18:07:01.336000 [ 643 ] DEBUG : Executing query drop database if exists re sync;drop table if exists system.a; on node2 (cluster.py:3658, query) 2026-04-10 18:07:01.956000 [ 643 ] DEBUG : Executing query drop database if exists re sync;drop table if exists system.a; on reading_node (cluster.py:3658, query) ______________ test_rename_distributed_parallel_insert_and_select ______________ [gw2] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = def test_rename_distributed_parallel_insert_and_select(started_cluster): table_name = "test_rename_distributed_parallel_insert_and_select" try: create_distributed_table(node1, table_name) insert(node1, table_name, 1000) p = Pool(15) tasks = [] for i in range(1): tasks.append( p.apply_async( rename_column_on_cluster, (node1, table_name, "num2", "foo2", 3, True), ) ) tasks.append( p.apply_async( rename_column_on_cluster, (node1, "%s_replicated" % table_name, "num2", "foo2", 3, True), ) ) tasks.append( p.apply_async( rename_column_on_cluster, (node1, table_name, "foo2", "foo3", 3, True), ) ) tasks.append( p.apply_async( rename_column_on_cluster, (node1, "%s_replicated" % table_name, "foo2", "foo3", 3, True), ) ) tasks.append( p.apply_async( rename_column_on_cluster, (node1, table_name, "foo3", "num2", 3, True), ) ) tasks.append( p.apply_async( rename_column_on_cluster, (node1, "%s_replicated" % table_name, "foo3", "num2", 3, True), ) ) tasks.append( p.apply_async(insert, (node1, table_name, 10, ["num", "foo3"], 5, True)) ) tasks.append( p.apply_async(insert, (node2, table_name, 10, ["num", "num2"], 5, True)) ) tasks.append( p.apply_async(insert, (node3, table_name, 10, ["num", "foo2"], 5, True)) ) tasks.append( p.apply_async(select, (node1, table_name, "foo2", None, 5, True)) ) tasks.append( p.apply_async(select, (node2, table_name, "foo3", None, 5, True)) ) tasks.append( p.apply_async(select, (node3, table_name, "num2", None, 5, True)) ) for task in tasks: > task.get(timeout=240) test_rename_column/test.py:810: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 240 def get(self, timeout=None): self.wait(timeout) if not self.ready(): > raise TimeoutError E multiprocessing.context.TimeoutError /usr/lib/python3.10/multiprocessing/pool.py:770: TimeoutError During handling of the above exception, another exception occurred: started_cluster = def test_rename_distributed_parallel_insert_and_select(started_cluster): table_name = "test_rename_distributed_parallel_insert_and_select" try: create_distributed_table(node1, table_name) insert(node1, table_name, 1000) p = Pool(15) tasks = [] for i in range(1): tasks.append( p.apply_async( rename_column_on_cluster, (node1, table_name, "num2", "foo2", 3, True), ) ) tasks.append( p.apply_async( rename_column_on_cluster, (node1, "%s_replicated" % table_name, "num2", "foo2", 3, True), ) ) tasks.append( p.apply_async( rename_column_on_cluster, (node1, table_name, "foo2", "foo3", 3, True), ) ) tasks.append( p.apply_async( rename_column_on_cluster, (node1, "%s_replicated" % table_name, "foo2", "foo3", 3, True), ) ) tasks.append( p.apply_async( rename_column_on_cluster, (node1, table_name, "foo3", "num2", 3, True), ) ) tasks.append( p.apply_async( rename_column_on_cluster, (node1, "%s_replicated" % table_name, "foo3", "num2", 3, True), ) ) tasks.append( p.apply_async(insert, (node1, table_name, 10, ["num", "foo3"], 5, True)) ) tasks.append( p.apply_async(insert, (node2, table_name, 10, ["num", "num2"], 5, True)) ) tasks.append( p.apply_async(insert, (node3, table_name, 10, ["num", "foo2"], 5, True)) ) tasks.append( p.apply_async(select, (node1, table_name, "foo2", None, 5, True)) ) tasks.append( p.apply_async(select, (node2, table_name, "foo3", None, 5, True)) ) tasks.append( p.apply_async(select, (node3, table_name, "num2", None, 5, True)) ) for task in tasks: task.get(timeout=240) rename_column_on_cluster(node1, table_name, "foo2", "num2", 1, True) rename_column_on_cluster( node1, "%s_replicated" % table_name, "foo2", "num2", 1, True ) rename_column_on_cluster(node1, table_name, "foo3", "num2", 1, True) rename_column_on_cluster( node1, "%s_replicated" % table_name, "foo3", "num2", 1, True ) insert(node1, table_name, 1000, col_names=["num", "num2"]) select(node1, table_name, "num2") select(node2, table_name, "num2") select(node3, table_name, "num2") select(node4, table_name, "num2") finally: > drop_distributed_table(node1, table_name) test_rename_column/test.py:827: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test_rename_column/test.py:122: in drop_distributed_table node.query( helpers/cluster.py:3659: in query return self.client.query( helpers/client.py:39: in wrap return func(self, *args, **kwargs) helpers/client.py:79: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 159, stderr: Received exception from server (version 25.3.8): E Code: 159. DB::Exception: Received from 172.16.9.7:9000. DB::Exception: Distributed DDL task /clickhouse/task_queue/ddl/query-0000000021 is not finished on 2 of 4 hosts (0 of them are currently executing the task, 0 are inactive). They are going to execute the query in background. Was waiting for 180.849639162 seconds, which is longer than distributed_ddl_task_timeout. Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x0000000036f39bb1 E 1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001b1bca51 E 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000b7f426b E 3. ./src/Common/Exception.h:130: DB::DDLOnClusterQueryStatusSource::handleTimeoutExceeded() @ 0x0000000029fffb09 E 4. ./build_docker/./src/Interpreters/DistributedQueryStatusSource.cpp:0: DB::DistributedQueryStatusSource::generate() @ 0x0000000028c1b7ef E 5. ./build_docker/./src/Processors/ISource.cpp:139: DB::ISource::tryGenerate() @ 0x000000002f38c397 E 6. ./build_docker/./src/Processors/ISource.cpp:108: DB::ISource::work() @ 0x000000002f38b984 E 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:53: DB::ExecutionThreadContext::executeTask() @ 0x000000002f3cb734 E 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:296: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002f3ac280 E 9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:262: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002f3aa7fc E 10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:128: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002f3a9fd8 E 11. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:76: void std::__function::__policy_invoker::__call_impl[abi:ne190107]::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002f3d388a E 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b486e11 E 13. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001b495270 E 14. asan_thread_start(void*) @ 0x000000000b7a6e77 E 15. ? @ 0x00007f92bdcecac3 E 16. ? @ 0x00007f92bdd7e850 E E Job's origin stack trace: E 0. ./build_docker/./src/Common/StackTrace.cpp:386: StackTrace::StackTrace() @ 0x000000001b3230c7 E 1. ./build_docker/./src/Common/ThreadPool.cpp:130: void boost::heap::priority_queue, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::emplace, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const, bool&, (anonymous namespace)::ScopedDecrement>(std::function&&, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const&&, bool&, (anonymous namespace)::ScopedDecrement&&) @ 0x000000001b4853c4 E 2. ./build_docker/./src/Common/ThreadPool.cpp:401: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001b489c6a E 3. ./build_docker/./src/Common/ThreadPool.cpp:506: ThreadPoolImpl::scheduleOrThrow(std::function, Priority, unsigned long, bool) @ 0x000000001b48a6dd E 4. ./src/Common/ThreadPool.h:279: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000002f3d14a7 E 5. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:131: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002f3d1ebc E 6. ./build_docker/./src/Server/TCPHandler.cpp:1237: DB::TCPHandler::processOrdinaryQuery(DB::QueryState&) @ 0x000000002f1b5fdc E 7. ./build_docker/./src/Server/TCPHandler.cpp:678: DB::TCPHandler::runImpl() @ 0x000000002f19afe0 E 8. ./build_docker/./src/Server/TCPHandler.cpp:2629: DB::TCPHandler::run() @ 0x000000002f1daa5a E 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000037114faf E 10. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000037115c77 E 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x00000000370244cb E 12. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003701e0c8 E 13. asan_thread_start(void*) @ 0x000000000b7a6e77 E 14. ? @ 0x00007f92bdcecac3 E 15. ? @ 0x00007f92bdd7e850 E . (TIMEOUT_EXCEEDED) E (query: DROP TABLE IF EXISTS test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster SYNC) helpers/client.py:248: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-10 18:02:13.411000 [ 640 ] DEBUG : Executing query CREATE TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/{shard}/test_rename_distributed_parallel_insert_and_select_replicated', '{replica}') ORDER BY num PARTITION BY num % 100; on node1 (cluster.py:3658, query) 2026-04-10 18:02:14.681000 [ 640 ] DEBUG : Executing query CREATE TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster AS test_rename_distributed_parallel_insert_and_select_replicated ENGINE = Distributed(test_cluster, default, test_rename_distributed_parallel_insert_and_select_replicated, rand()) on node1 (cluster.py:3658, query) 2026-04-10 18:02:15.552000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(1000) on node1 (cluster.py:3658, query) 2026-04-10 18:02:31.217000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 (cluster.py:3658, query) 2026-04-10 18:02:31.220000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 (cluster.py:3658, query) 2026-04-10 18:02:31.220000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo2 to foo3 on node1 (cluster.py:3658, query) 2026-04-10 18:02:31.220000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo2 to foo3 on node1 (cluster.py:3658, query) 2026-04-10 18:02:31.220000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 (cluster.py:3658, query) 2026-04-10 18:02:31.223000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 (cluster.py:3658, query) 2026-04-10 18:02:31.228000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(10) on node1 (cluster.py:3658, query) 2026-04-10 18:02:31.230000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(10) on node2 (cluster.py:3658, query) 2026-04-10 18:02:31.231000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(10) on node3 (cluster.py:3658, query) 2026-04-10 18:02:31.235000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo2 % 1000 > 0 on node1 (cluster.py:3658, query) 2026-04-10 18:02:31.245000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo3 % 1000 > 0 on node2 (cluster.py:3658, query) 2026-04-10 18:02:31.246000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node3 (cluster.py:3658, query) 2026-04-10 18:02:36.090000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(10) on node1 (cluster.py:3658, query) 2026-04-10 18:02:36.166000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo2 % 1000 > 0 on node1 (cluster.py:3658, query) 2026-04-10 18:02:37.062000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo2 % 1000 > 0 on node1 (cluster.py:3658, query) 2026-04-10 18:02:37.202000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(10) on node3 (cluster.py:3658, query) 2026-04-10 18:02:37.385000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(10) on node1 (cluster.py:3658, query) 2026-04-10 18:02:37.887000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo3 % 1000 > 0 on node2 (cluster.py:3658, query) 2026-04-10 18:02:38.070000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(10) on node1 (cluster.py:3658, query) 2026-04-10 18:02:38.110000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo2 % 1000 > 0 on node1 (cluster.py:3658, query) 2026-04-10 18:02:38.214000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo3 % 1000 > 0 on node2 (cluster.py:3658, query) 2026-04-10 18:02:38.468000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(10) on node3 (cluster.py:3658, query) 2026-04-10 18:02:38.866000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo3 % 1000 > 0 on node2 (cluster.py:3658, query) 2026-04-10 18:02:39.060000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo2 % 1000 > 0 on node1 (cluster.py:3658, query) 2026-04-10 18:02:39.072000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(10) on node1 (cluster.py:3658, query) 2026-04-10 18:02:39.076000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(10) on node3 (cluster.py:3658, query) 2026-04-10 18:02:39.245000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo3 % 1000 > 0 on node2 (cluster.py:3658, query) 2026-04-10 18:02:39.600000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(10) on node2 (cluster.py:3658, query) 2026-04-10 18:02:39.736000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(10) on node3 (cluster.py:3658, query) 2026-04-10 18:02:40.192000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo2 to foo3 on node1 (cluster.py:3658, query) 2026-04-10 18:02:44.516000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node3 (cluster.py:3658, query) 2026-04-10 18:02:44.715000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(10) on node2 (cluster.py:3658, query) 2026-04-10 18:02:45.610000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(10) on node2 (cluster.py:3658, query) 2026-04-10 18:02:46.186000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo2 to foo3 on node1 (cluster.py:3658, query) 2026-04-10 18:02:46.458000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node3 (cluster.py:3658, query) 2026-04-10 18:02:46.870000 [ 640 ] DEBUG : Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(10) on node2 (cluster.py:3658, query) 2026-04-10 18:02:47.306000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node3 (cluster.py:3658, query) 2026-04-10 18:02:48.160000 [ 640 ] DEBUG : Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node3 (cluster.py:3658, query) 2026-04-10 18:03:03.461000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 (cluster.py:3658, query) 2026-04-10 18:05:36.560000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo2 to foo3 on node1 (cluster.py:3658, query) 2026-04-10 18:05:36.911000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 (cluster.py:3658, query) 2026-04-10 18:05:36.911000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 (cluster.py:3658, query) 2026-04-10 18:05:36.996000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 (cluster.py:3658, query) 2026-04-10 18:06:04.950000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 (cluster.py:3658, query) 2026-04-10 18:06:31.215000 [ 640 ] DEBUG : Executing query DROP TABLE IF EXISTS test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster SYNC on node1 (cluster.py:3658, query) 2026-04-10 18:08:37.364000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 (cluster.py:3658, query) 2026-04-10 18:08:37.383000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 (cluster.py:3658, query) 2026-04-10 18:08:37.383000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 (cluster.py:3658, query) 2026-04-10 18:08:37.490000 [ 640 ] DEBUG : Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo2 to foo3 on node1 (cluster.py:3658, query) =============================== warnings summary =============================== test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy /usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host '172.16.8.5'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ----------------- generated report log file: parallel2_0.jsonl ----------------- ============================== slowest durations =============================== 439.01s call test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select 353.11s call test_refreshable_mv/test.py::test_backup_outer_table 136.49s call test_refreshable_mv/test.py::test_refresh_vs_shutdown_smoke 130.00s call test_rename_column/test.py::test_rename_distributed 96.06s call test_s3_plain_rewritable/test.py::test[s3_plain_rewritable-data/] 87.96s setup test_restart_with_unavailable_azure/test.py::test_cluster_alive_after_restart 63.85s call test_refreshable_mv/test.py::test_refreshable_mv_in_replicated_db 62.81s call test_recompression_ttl/test.py::test_recompression_multiple_ttls 61.30s call test_s3_plain_rewritable/test.py::test[s3_plain_rewritable_with_metadata_cache-data_with_cache/] 60.39s call test_restore_external_engines/test.py::test_restore_table_null 60.12s call test_recompression_ttl/test.py::test_recompression_simple 57.11s setup test_replicated_merge_tree_thread_schedule_timeouts/test.py::test_query_exception_on_thread_pool_full 53.24s call test_s3_plain_rewritable/test.py::test[cache_s3_plain_rewritable-data/] 43.89s setup test_restore_external_engines/test.py::test_restore_table 43.80s call test_refreshable_mv/test.py::test_backup_inner_table 42.14s call test_restore_external_engines/test.py::test_restore_table 40.02s call test_recompression_ttl/test.py::test_recompression_replicated 38.93s call test_role/test.py::test_roles_cache 37.83s teardown test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy 36.50s setup test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3] 31.93s teardown test_recompression_ttl/test.py::test_recompression_simple 31.31s call test_rocksdb_read_only/test.py::test_dirctory_missing_after_stop 29.24s teardown test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3_with_resolver] 28.66s setup test_prometheus_endpoint/test.py::test_prometheus_endpoint 28.25s setup test_rocksdb_read_only/test.py::test_dirctory_missing_after_stop 26.86s teardown test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case4] 26.60s setup test_replication_without_zookeeper/test.py::test_startup_without_zookeeper 25.84s teardown test_role/test.py::test_set_role 25.14s call test_role/test.py::test_revoke_requires_admin_option 24.91s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_17] 24.77s call test_role/test.py::test_introspection 24.42s teardown test_refreshable_mv/test.py::test_refreshable_mv_in_system_db 23.76s setup test_refreshable_mv/test.py::test_adding_replica 23.57s setup test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case1] 23.45s call test_role/test.py::test_role_expiration[True] 23.21s call test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3_with_resolver] 23.06s call test_refreshable_mv/test.py::test_pause 22.98s setup test_profile_settings_and_constraints_order/test.py::test_profile_settings_and_constraints_order 22.57s setup test_recompression_ttl/test.py::test_recompression_multiple_ttls 22.35s setup test_rename_column/test.py::test_rename_distributed 21.98s teardown test_s3_plain_rewritable/test.py::test[s3_plain_rewritable_with_metadata_cache-data_with_cache/] 21.59s teardown test_restart_with_unavailable_azure/test.py::test_cluster_alive_after_restart 21.11s teardown test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_path_ambiguity 20.59s setup test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_no_proxy 19.20s setup test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_auxiliary_zookeeper 18.57s call test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list_no_proxy 18.32s call test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_no_proxy 17.89s setup test_s3_plain_rewritable/test.py::test[cache_s3_plain_rewritable-data/] 17.55s call test_role/test.py::test_role_expiration[False] 16.98s call test_replication_without_zookeeper/test.py::test_startup_without_zookeeper 16.97s teardown test_restore_external_engines/test.py::test_restore_table_null 16.36s call test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3] 15.85s teardown test_profile_settings_and_constraints_order/test.py::test_profile_settings_and_constraints_order 15.74s call test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_drop_replicated_merge_tree_with_auxiliary_zookeeper 15.55s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_6] 15.44s call test_refreshable_mv/test.py::test_refreshable_mv_in_system_db 15.35s call test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list 15.16s call test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy 14.74s call test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy 14.37s setup test_role/test.py::test_admin_option 14.07s setup test_remote_prewhere/test.py::test_remote 12.62s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_23] 12.61s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_4] 12.31s call test_mysql57_database_engine/test.py::test_restart_server 12.02s call test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_no_proxy 11.44s call test_refreshable_mv/test.py::test_refreshable_mv_in_read_only_node 10.94s call test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case4] 10.64s call test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case2] 10.50s teardown test_prometheus_endpoint/test.py::test_prometheus_endpoint 10.40s call test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_default_zookeeper 10.38s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_2] 10.12s call test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1] 9.81s call test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2] 9.77s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_9] 9.65s call test_mysql57_database_engine/test.py::test_mysql_types[datetime_default] 9.60s teardown test_rename_column/test.py::test_rename_with_parallel_ttl_move 9.46s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_20] 9.21s call test_replicated_merge_tree_thread_schedule_timeouts/test.py::test_query_exception_on_thread_pool_full 8.95s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_24] 8.62s call test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_path_ambiguity 8.59s call test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_auxiliary_zookeeper 7.98s call test_refreshable_mv/test.py::test_adding_replica 7.24s call test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case3] 7.21s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_18] 7.19s call test_restart_with_unavailable_azure/test.py::test_cluster_alive_after_restart 7.16s teardown test_replicated_merge_tree_thread_schedule_timeouts/test.py::test_query_exception_on_thread_pool_full 7.13s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_3] 6.94s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_7] 6.55s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_81] 6.41s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_1] 5.96s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_22] 5.87s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_5] 5.83s teardown test_refreshable_mv/test.py::test_backup_inner_table 5.66s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_21] 5.53s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_19] 5.45s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_25] 5.44s call test_role/test.py::test_function_current_roles 5.36s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_80] 5.19s call test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3] 5.07s teardown test_mysql57_database_engine/test.py::test_restart_server 5.06s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1] 4.74s call test_role/test.py::test_grant_role_to_role 4.68s teardown test_role/test.py::test_roles_cache 4.42s teardown test_remote_prewhere/test.py::test_remote 4.01s teardown test_refreshable_mv/test.py::test_backup_outer_table 3.99s call test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case1] 3.97s teardown test_role/test.py::test_role_expiration[True] 3.95s call test_role/test.py::test_admin_option 3.94s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_17] 3.85s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2] 3.72s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_1] 3.62s call test_role/test.py::test_set_role 3.56s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_2] 3.46s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_3] 3.39s call test_rocksdb_read_only/test.py::test_read_only 3.35s call test_mysql57_database_engine/test.py::test_predefined_connection_configuration 3.22s call test_role/test.py::test_create_role 3.15s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6] 3.14s call test_prometheus_endpoint/test.py::test_prometheus_endpoint 3.06s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_4] 3.05s call test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default] 2.97s call test_role/test.py::test_combine_privileges 2.94s call test_mysql57_database_engine/test.py::test_mysql_types[float_1] 2.90s call test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6] 2.89s call test_mysql57_database_engine/test.py::test_mysql_types[float_2] 2.89s teardown test_restore_external_engines/test.py::test_restore_table 2.41s teardown test_replication_without_zookeeper/test.py::test_startup_without_zookeeper 2.41s teardown test_refreshable_mv/test.py::test_adding_replica 2.34s call test_role/test.py::test_changing_default_roles_affects_new_sessions_only 2.08s call test_profile_settings_and_constraints_order/test.py::test_profile_settings_and_constraints_order 1.60s teardown test_rocksdb_read_only/test.py::test_read_only 1.16s teardown test_role/test.py::test_create_role 1.11s teardown test_role/test.py::test_role_expiration[False] 1.08s teardown test_role/test.py::test_function_current_roles 1.07s call test_mysql57_database_engine/test.py::test_password_leak 1.07s teardown test_role/test.py::test_revoke_requires_admin_option 1.03s teardown test_role/test.py::test_grant_role_to_role 1.00s teardown test_role/test.py::test_introspection 0.93s call test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_not_exists_auxiliary_zookeeper 0.77s teardown test_role/test.py::test_changing_default_roles_affects_new_sessions_only 0.75s teardown test_refreshable_mv/test.py::test_pause 0.73s teardown test_role/test.py::test_admin_option 0.54s teardown test_role/test.py::test_combine_privileges 0.37s call test_rename_column/test.py::test_rename_with_parallel_slow_insert 0.37s call test_rename_column/test.py::test_rename_with_parallel_insert 0.32s call test_rename_column/test.py::test_rename_with_parallel_merges 0.32s call test_remote_prewhere/test.py::test_remote 0.32s call test_rename_column/test.py::test_rename_with_parallel_select 0.32s call test_rename_column/test.py::test_rename_parallel 0.32s call test_rename_column/test.py::test_rename_parallel_same_node 0.32s call test_rename_column/test.py::test_rename_with_parallel_ttl_delete 0.27s call test_rename_column/test.py::test_rename_with_parallel_ttl_move 0.05s teardown test_recompression_ttl/test.py::test_recompression_multiple_ttls 0.01s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_17] 0.01s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_23] 0.01s setup test_role/test.py::test_role_expiration[True] 0.01s setup test_role/test.py::test_set_role 0.01s teardown test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case2] 0.01s teardown test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_no_proxy 0.01s teardown test_s3_plain_rewritable/test.py::test[cache_s3_plain_rewritable-data/] 0.01s teardown test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list_no_proxy 0.01s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_3] 0.01s setup test_s3_plain_rewritable/test.py::test[s3_plain_rewritable_with_metadata_cache-data_with_cache/] 0.01s setup test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case2] 0.01s teardown test_s3_plain_rewritable/test.py::test[s3_plain_rewritable-data/] 0.00s setup test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case3] 0.00s setup test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list 0.00s teardown test_rename_column/test.py::test_rename_distributed 0.00s setup test_role/test.py::test_roles_cache 0.00s setup test_role/test.py::test_role_expiration[False] 0.00s setup test_restore_external_engines/test.py::test_restore_table_null 0.00s setup test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_drop_replicated_merge_tree_with_auxiliary_zookeeper 0.00s teardown test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3] 0.00s setup test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_23] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_24] 0.00s setup test_refreshable_mv/test.py::test_backup_outer_table 0.00s setup test_recompression_ttl/test.py::test_recompression_replicated 0.00s setup test_refreshable_mv/test.py::test_backup_inner_table 0.00s setup test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_no_proxy 0.00s setup test_s3_plain_rewritable/test.py::test[s3_plain_rewritable-data/] 0.00s setup test_role/test.py::test_revoke_requires_admin_option 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3] 0.00s setup test_role/test.py::test_create_role 0.00s teardown test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_drop_replicated_merge_tree_with_auxiliary_zookeeper 0.00s teardown test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_no_proxy 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_4] 0.00s setup test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_path_ambiguity 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_80] 0.00s setup test_role/test.py::test_function_current_roles 0.00s setup test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2] 0.00s setup test_rocksdb_read_only/test.py::test_read_only 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2] 0.00s teardown test_rocksdb_read_only/test.py::test_dirctory_missing_after_stop 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_20] 0.00s setup test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6] 0.00s setup test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_not_exists_auxiliary_zookeeper 0.00s teardown test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_22] 0.00s setup test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list_no_proxy 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_18] 0.00s setup test_refreshable_mv/test.py::test_pause 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_5] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_21] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_80] 0.00s setup test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3_with_resolver] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_81] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_2] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_3] 0.00s teardown test_refreshable_mv/test.py::test_refresh_vs_shutdown_smoke 0.00s teardown test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case1] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_7] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[float_2] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_4] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[float_1] 0.00s setup test_mysql57_database_engine/test.py::test_password_leak 0.00s teardown test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_default_zookeeper 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_2] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_1] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[datetime_default] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_6] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_9] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_19] 0.00s teardown test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select 0.00s teardown test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case3] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_4] 0.00s setup test_role/test.py::test_grant_role_to_role 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_3] 0.00s setup test_refreshable_mv/test.py::test_refreshable_mv_in_system_db 0.00s setup test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case4] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_2] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_1] 0.00s setup test_role/test.py::test_combine_privileges 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_25] 0.00s teardown test_refreshable_mv/test.py::test_refreshable_mv_in_replicated_db 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_3] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_6] 0.00s setup test_rename_column/test.py::test_rename_parallel_same_node 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_19] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_9] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_25] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_81] 0.00s setup test_refreshable_mv/test.py::test_refresh_vs_shutdown_smoke 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_22] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[datetime_default] 0.00s setup test_rename_column/test.py::test_rename_parallel 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_24] 0.00s teardown test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_not_exists_auxiliary_zookeeper 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_20] 0.00s setup test_mysql57_database_engine/test.py::test_predefined_connection_configuration 0.00s setup test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_default_zookeeper 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_7] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1] 0.00s teardown test_refreshable_mv/test.py::test_refreshable_mv_in_read_only_node 0.00s setup test_refreshable_mv/test.py::test_refreshable_mv_in_replicated_db 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_21] 0.00s teardown test_recompression_ttl/test.py::test_recompression_replicated 0.00s setup test_rename_column/test.py::test_rename_with_parallel_insert 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[float_2] 0.00s setup test_role/test.py::test_introspection 0.00s setup test_refreshable_mv/test.py::test_refreshable_mv_in_read_only_node 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[float_1] 0.00s setup test_rename_column/test.py::test_rename_with_parallel_slow_insert 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_18] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_4] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3] 0.00s setup test_rename_column/test.py::test_rename_with_parallel_select 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_1] 0.00s teardown test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_auxiliary_zookeeper 0.00s teardown test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy 0.00s setup test_role/test.py::test_changing_default_roles_affects_new_sessions_only 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_1] 0.00s setup test_rename_column/test.py::test_rename_with_parallel_ttl_delete 0.00s teardown test_mysql57_database_engine/test.py::test_password_leak 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6] 0.00s setup test_mysql57_database_engine/test.py::test_restart_server 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_2] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_5] 0.00s teardown test_rename_column/test.py::test_rename_with_parallel_select 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2] 0.00s setup test_recompression_ttl/test.py::test_recompression_simple 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1] 0.00s teardown test_rename_column/test.py::test_rename_parallel_same_node 0.00s setup test_rename_column/test.py::test_rename_with_parallel_merges 0.00s setup test_rename_column/test.py::test_rename_with_parallel_ttl_move 0.00s teardown test_mysql57_database_engine/test.py::test_predefined_connection_configuration 0.00s teardown test_rename_column/test.py::test_rename_parallel 0.00s teardown test_rename_column/test.py::test_rename_with_parallel_merges 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6] 0.00s teardown test_rename_column/test.py::test_rename_with_parallel_slow_insert 0.00s teardown test_rename_column/test.py::test_rename_with_parallel_ttl_delete 0.00s teardown test_rename_column/test.py::test_rename_with_parallel_insert =========================== short test summary info ============================ FAILED test_recompression_ttl/test.py::test_recompression_replicated - helper... FAILED test_rename_column/test.py::test_rename_distributed - AssertionError: ... FAILED test_refreshable_mv/test.py::test_backup_outer_table - helpers.client.... FAILED test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select PASSED test_role/test.py::test_admin_option PASSED test_role/test.py::test_changing_default_roles_affects_new_sessions_only PASSED test_role/test.py::test_combine_privileges PASSED test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case1] PASSED test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_auxiliary_zookeeper PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_17] PASSED test_role/test.py::test_create_role PASSED test_refreshable_mv/test.py::test_adding_replica PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_no_proxy PASSED test_role/test.py::test_function_current_roles PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_18] PASSED test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_default_zookeeper PASSED test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case2] PASSED test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_not_exists_auxiliary_zookeeper PASSED test_role/test.py::test_grant_role_to_role PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_19] PASSED test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case3] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_1] PASSED test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_drop_replicated_merge_tree_with_auxiliary_zookeeper PASSED test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case4] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_20] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_21] PASSED test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_path_ambiguity PASSED test_role/test.py::test_introspection PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_22] PASSED test_s3_plain_rewritable/test.py::test[cache_s3_plain_rewritable-data/] PASSED test_refreshable_mv/test.py::test_backup_inner_table PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list_no_proxy PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_23] PASSED test_recompression_ttl/test.py::test_recompression_multiple_ttls PASSED test_restore_external_engines/test.py::test_restore_table PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_24] PASSED test_role/test.py::test_revoke_requires_admin_option PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_25] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_no_proxy PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_2] PASSED test_profile_settings_and_constraints_order/test.py::test_profile_settings_and_constraints_order PASSED test_role/test.py::test_role_expiration[False] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_3] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_4] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_5] PASSED test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3] PASSED test_role/test.py::test_role_expiration[True] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_6] PASSED test_restore_external_engines/test.py::test_restore_table_null PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_7] PASSED test_prometheus_endpoint/test.py::test_prometheus_endpoint PASSED test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3_with_resolver] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_80] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_81] PASSED test_s3_plain_rewritable/test.py::test[s3_plain_rewritable-data/] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_9] PASSED test_role/test.py::test_roles_cache PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1] PASSED test_recompression_ttl/test.py::test_recompression_simple PASSED test_role/test.py::test_set_role PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_default] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1] PASSED test_replicated_merge_tree_thread_schedule_timeouts/test.py::test_query_exception_on_thread_pool_full PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_1] PASSED test_rocksdb_read_only/test.py::test_dirctory_missing_after_stop PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_2] PASSED test_s3_plain_rewritable/test.py::test[s3_plain_rewritable_with_metadata_cache-data_with_cache/] PASSED test_rocksdb_read_only/test.py::test_read_only PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_3] PASSED test_replication_without_zookeeper/test.py::test_startup_without_zookeeper PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_4] PASSED test_mysql57_database_engine/test.py::test_mysql_types[float_1] PASSED test_mysql57_database_engine/test.py::test_mysql_types[float_2] PASSED test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6] PASSED test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default] PASSED test_mysql57_database_engine/test.py::test_password_leak PASSED test_mysql57_database_engine/test.py::test_predefined_connection_configuration PASSED test_restart_with_unavailable_azure/test.py::test_cluster_alive_after_restart PASSED test_remote_prewhere/test.py::test_remote PASSED test_mysql57_database_engine/test.py::test_restart_server PASSED test_refreshable_mv/test.py::test_pause PASSED test_refreshable_mv/test.py::test_refresh_vs_shutdown_smoke PASSED test_refreshable_mv/test.py::test_refreshable_mv_in_read_only_node PASSED test_refreshable_mv/test.py::test_refreshable_mv_in_replicated_db PASSED test_refreshable_mv/test.py::test_refreshable_mv_in_system_db SKIPPED [1] test_rename_column/test.py:338: Consume tons of memory with sanitizer SKIPPED [1] test_rename_column/test.py:297: Consume tons of memory with sanitizer SKIPPED [1] test_rename_column/test.py:433: Consume tons of memory with sanitizer SKIPPED [1] test_rename_column/test.py:489: Consume tons of memory with sanitizer SKIPPED [1] test_rename_column/test.py:380: Consume tons of memory with sanitizer SKIPPED [1] test_rename_column/test.py:555: Consume tons of memory with sanitizer SKIPPED [1] test_rename_column/test.py:670: Consume tons of memory with sanitizer SKIPPED [1] test_rename_column/test.py:596: Consume tons of memory with sanitizer ======= 4 failed, 88 passed, 8 skipped, 7 warnings in 724.44s (0:12:04) ======== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 492, in subprocess.check_call(cmd, shell=True, bufsize=0) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_vgayu6 --privileged --dns-search='.' --memory=30709035008 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e CLICKHOUSE_USE_OLD_ANALYZER=1 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel2_0.jsonl --report-log-exclude-logs-on-passed-tests 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_17]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_18]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_19]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_20]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_21]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_22]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_23]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_24]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_25]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_3]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_4]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_5]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_6]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_7]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_80]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_81]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_9]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_default]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_3]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_4]' 'test_mysql57_database_engine/test.py::test_mysql_types[float_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[float_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6]' 'test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default]' test_mysql57_database_engine/test.py::test_password_leak test_mysql57_database_engine/test.py::test_predefined_connection_configuration test_mysql57_database_engine/test.py::test_restart_server test_profile_settings_and_constraints_order/test.py::test_profile_settings_and_constraints_order test_prometheus_endpoint/test.py::test_prometheus_endpoint test_recompression_ttl/test.py::test_recompression_multiple_ttls test_recompression_ttl/test.py::test_recompression_replicated test_recompression_ttl/test.py::test_recompression_simple test_refreshable_mv/test.py::test_adding_replica test_refreshable_mv/test.py::test_backup_inner_table test_refreshable_mv/test.py::test_backup_outer_table test_refreshable_mv/test.py::test_pause test_refreshable_mv/test.py::test_refresh_vs_shutdown_smoke test_refreshable_mv/test.py::test_refreshable_mv_in_read_only_node test_refreshable_mv/test.py::test_refreshable_mv_in_replicated_db test_refreshable_mv/test.py::test_refreshable_mv_in_system_db test_remote_prewhere/test.py::test_remote test_rename_column/test.py::test_rename_distributed test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select test_rename_column/test.py::test_rename_parallel test_rename_column/test.py::test_rename_parallel_same_node test_rename_column/test.py::test_rename_with_parallel_insert test_rename_column/test.py::test_rename_with_parallel_merges test_rename_column/test.py::test_rename_with_parallel_select test_rename_column/test.py::test_rename_with_parallel_slow_insert test_rename_column/test.py::test_rename_with_parallel_ttl_delete test_rename_column/test.py::test_rename_with_parallel_ttl_move test_replicated_merge_tree_thread_schedule_timeouts/test.py::test_query_exception_on_thread_pool_full test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_auxiliary_zookeeper test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_default_zookeeper test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_create_replicated_merge_tree_with_not_exists_auxiliary_zookeeper test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_drop_replicated_merge_tree_with_auxiliary_zookeeper test_replicated_merge_tree_with_auxiliary_zookeepers/test.py::test_path_ambiguity test_replication_without_zookeeper/test.py::test_startup_without_zookeeper test_restart_with_unavailable_azure/test.py::test_cluster_alive_after_restart test_restore_external_engines/test.py::test_restore_table test_restore_external_engines/test.py::test_restore_table_null test_rocksdb_read_only/test.py::test_dirctory_missing_after_stop test_rocksdb_read_only/test.py::test_read_only test_role/test.py::test_admin_option test_role/test.py::test_changing_default_roles_affects_new_sessions_only test_role/test.py::test_combine_privileges test_role/test.py::test_create_role test_role/test.py::test_function_current_roles test_role/test.py::test_grant_role_to_role test_role/test.py::test_introspection test_role/test.py::test_revoke_requires_admin_option 'test_role/test.py::test_role_expiration[False]' 'test_role/test.py::test_role_expiration[True]' test_role/test.py::test_roles_cache test_role/test.py::test_set_role 'test_s3_plain_rewritable/test.py::test[cache_s3_plain_rewritable-data/]' 'test_s3_plain_rewritable/test.py::test[s3_plain_rewritable-data/]' 'test_s3_plain_rewritable/test.py::test[s3_plain_rewritable_with_metadata_cache-data_with_cache/]' 'test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3]' 'test_s3_storage_conf_proxy/test.py::test_s3_with_proxy_list[s3_with_resolver]' test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_no_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy 'test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case1]' 'test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case2]' 'test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case3]' 'test_send_request_to_leader_replica/test.py::test_alter_table_drop_partition[case4]' -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 ' returned non-zero exit status 1.